5 research outputs found
Adaptive Grey-Box Fuzz-Testing with Thompson Sampling
Fuzz testing, or "fuzzing," refers to a widely deployed class of techniques
for testing programs by generating a set of inputs for the express purpose of
finding bugs and identifying security flaws. Grey-box fuzzing, the most popular
fuzzing strategy, combines light program instrumentation with a data driven
process to generate new program inputs. In this work, we present a machine
learning approach that builds on AFL, the preeminent grey-box fuzzer, by
adaptively learning a probability distribution over its mutation operators on a
program-specific basis. These operators, which are selected uniformly at random
in AFL and mutational fuzzers in general, dictate how new inputs are generated,
a core part of the fuzzer's efficacy. Our main contributions are two-fold:
First, we show that a sampling distribution over mutation operators estimated
from training programs can significantly improve performance of AFL. Second, we
introduce a Thompson Sampling, bandit-based optimization approach that
fine-tunes the mutator distribution adaptively, during the course of fuzzing an
individual program. A set of experiments across complex programs demonstrates
that tuning the mutational operator distribution generates sets of inputs that
yield significantly higher code coverage and finds more crashes faster and more
reliably than both baseline versions of AFL as well as other AFL-based learning
approaches.Comment: Published as a workshop paper in the 11th ACM Workshop on Artificial
Intelligence and Security (AISec '18) with the 25th ACM Conference on
Computer and Communications Security (CCS '18
Semantic Fuzzing with Zest
Programs expecting structured inputs often consist of both a syntactic
analysis stage, which parses raw input, and a semantic analysis stage, which
conducts checks on the parsed input and executes the core logic of the program.
Generator-based testing tools in the lineage of QuickCheck are a promising way
to generate random syntactically valid test inputs for these programs. We
present Zest, a technique which automatically guides QuickCheck-like
randominput generators to better explore the semantic analysis stage of test
programs. Zest converts random-input generators into deterministic parametric
generators. We present the key insight that mutations in the untyped parameter
domain map to structural mutations in the input domain. Zest leverages program
feedback in the form of code coverage and input validity to perform
feedback-directed parameter search. We evaluate Zest against AFL and QuickCheck
on five Java programs: Maven, Ant, BCEL, Closure, and Rhino. Zest covers
1.03x-2.81x as many branches within the benchmarks semantic analysis stages as
baseline techniques. Further, we find 10 new bugs in the semantic analysis
stages of these benchmarks. Zest is the most effective technique in finding
these bugs reliably and quickly, requiring at most 10 minutes on average to
find each bug.Comment: To appear in Proceedings of 28th ACM SIGSOFT International Symposium
on Software Testing and Analysis (ISSTA'19
Harvey: A Greybox Fuzzer for Smart Contracts
We present Harvey, an industrial greybox fuzzer for smart contracts, which
are programs managing accounts on a blockchain. Greybox fuzzing is a
lightweight test-generation approach that effectively detects bugs and security
vulnerabilities. However, greybox fuzzers randomly mutate program inputs to
exercise new paths; this makes it challenging to cover code that is guarded by
narrow checks, which are satisfied by no more than a few input values.
Moreover, most real-world smart contracts transition through many different
states during their lifetime, e.g., for every bid in an auction. To explore
these states and thereby detect deep vulnerabilities, a greybox fuzzer would
need to generate sequences of contract transactions, e.g., by creating bids
from multiple users, while at the same time keeping the search space and test
suite tractable. In this experience paper, we explain how Harvey alleviates
both challenges with two key fuzzing techniques and distill the main lessons
learned. First, Harvey extends standard greybox fuzzing with a method for
predicting new inputs that are more likely to cover new paths or reveal
vulnerabilities in smart contracts. Second, it fuzzes transaction sequences in
a targeted and demand-driven way. We have evaluated our approach on 27
real-world contracts. Our experiments show that the underlying techniques
significantly increase Harvey's effectiveness in achieving high coverage and
detecting vulnerabilities, in most cases orders-of-magnitude faster; they also
reveal new insights about contract code.Comment: arXiv admin note: substantial text overlap with arXiv:1807.0787
Learning Input Tokens for Effective Fuzzing
Modern fuzzing tools like AFL operate at a lexical level: They explore the input space of tested programs one byte after another. For inputs with complex syntactical properties, this is very inefficient, as keywords and other tokens have to be composed one character at a time. Fuzzers thus allow to specify dictionaries listing possible tokens the input can be composed from; such dictionaries speed up fuzzers dramatically. Also, fuzzers make use of dynamic tainting to track input tokens and infer values that are expected in the input validation phase. Unfortunately, such tokens are usually implicitly converted to program specific values which causes a loss of the taints attached to the input data in the lexical phase.
In this paper we present a technique to extend dynamic tainting to not only track explicit data flows but also taint implicitly converted data without suffering from taint explosion. This extension makes it possible to augment existing techniques and automatically infer a set of tokens and seed inputs for the input language of a program given nothing but the source code. Specifically targeting the lexical analysis of an input processor, our lFuzzer test generator systematically explores branches of the lexical analysis, producing a set of tokens that fully cover all decisions seen. The resulting set of tokens can be directly used as a dictionary for fuzzing. Along with the token extraction seed inputs are generated which give further fuzzing processes a head start. In our experiments, the lFuzzer-AFL combination achieves up to 17% more coverage on complex input formats like JSON, LISP, tinyC, and JavaScript compared to AFL
Optimizing Seed Selection for Fuzzing
Randomly mutating well-formed program inputs or simply fuzzing, is a highly effective and widely used strategy to find bugs in software. Other than showing fuzzers find bugs, there has been little systematic effort in understanding the science of how to fuzz properly. In this paper, we focus on how to mathematically formulate and reason about one critical aspect in fuzzing: how best to pick seed files to maximize the total number of bugs found during a fuzz campaign. We design and evaluate six different algorithms using over 650 CPU days on Amazon Elastic Compute Cloud (EC2) to provide ground truth data. Overall, we find 240 bugs in 8 applications and show that the choice of algorithm can greatly increase the number of bugs found. We also show that current seed selection strategies as found in Peach may fare no better than picking seeds at random. We make our data set and code publicly available.</p